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ABSTRACT 

Each year the Defense Productivity Program Office 
(DPPO) disburses funds for Productivity Investment Projects 
(PIFs). The purpose of these projects is to increase produc- 
tivity within the Department of Defense (DoD). To enhance 
these efforts, DPPO requested a study to be conducted to 
determine if methods of risk or uncertainty will affect the 
results obtained by the current procedure. This study 
applies various principles of uncertainty to this procedure 
and examines their impact on the project rankings. A back- 
ground of DPPO and PIFs is presented together with discussion 
of risk and uncertainty techniques, as well as the economic 
indicators used in ranking projects. A model is then ex- 
plained which will introduce uncertainty into the present 
procedure. Results of the initial comparison and sensitivity 
analysis is revealed. Conclusions are drawn based on these 
results and recommendations concerning alternate procedures 


and possible further research are presented. 
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I. INTRODUCTION 


A. BACKGROUND 

As early as February 1969, the Department of Defense 
(DoD) began to place emphasis upon increasing efficiency 
and productivity within its department. This was also 
complementary to the overall efforts of the federal govern- 
ment. These efforts led to the creation of the Defense 
Productivity Program (DPP) whose primary objective is "to 
achieve optimum productivity growth (increase the amount 
of goods produced or services rendered in relation to the 
amount of resources expended) throughout DoD" [6: pg. 1]. 
It should be noted that this does not include contractors. 
It is intended solely for DoD and the Service Components. 
Furthermore, this approach called for the development of 
Productivity Enhancing Capital Investments (PECIs). Responsi- 
bility for the administration of the program was assigned to 
the Assistant Secretary of Defense; Manpower, Reserve Affairs 
and Logistics (ASD(MRA&L)) who designated his authority to 
the Defense Productivity Program Office (DPPO). This Office 
acts as the coordinator for the DPP and serves as liaison 
between the Office of the Secretary of Defense (OSD) and 
Service Components in productivity matters. 

One of the problems faced by DPPO in the management of 


PECIs was the process of allocating funds. Due to budget 





constraints, not all of the recommended productivity projects 
could receive funds. To alleviate this dilemma, projects 
are ranked according to three economic indicators: internal 
rate of return (IRR); return on investment (ROI); and invest- 
ment per manpower space saved (INVPERSV). Projects then 
receive funding according to their rank until the budget 


for that fiscal year is exhausted. 


B. STATEMENT OF THE PROBLEM 

DPPO's concern is to ensure that funds are allotted 
to the most productive projects. Considering that projects 
forecast uncertain savings and costs into the future, this 
is an understandable concern. Under the present procedure, 
DPPO can only validate the use of correct discount rates in 
making these predictions. However, the realization of these 
forecasted costs and savings is questionable. The introduc- 
tion of methods involving risk and uncertainty may aid in 
the efficient allocation of funds. The question to be 
addressed in this thesis is whether or not the use of risk 
and uncertainty techniques will significantly change the 


current ranking of projects. 


C. OBJECTIVES OF THE ANALYSIS 

The objectives of this analysis will be to develop a 
model incorporating uncertainty, rank the projects using the 
model, and compare these results to the current ranked list. 


Since the present method ranks projects according to IRR, 
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ROI, and INVPERSV, the model will apply risk factors to these 
areas. The model will not introduce any new variables (e.g., 
net present value, payback period, etc.) which might affect 
the rankings. Furthermore, only those equations used by 
DPPO to calculate economic indicators will be presented in 
the model unless a simpler formula exists that portrays 
identical behavior. This specifically refers to the method 
used to compute IRR. The current method uses average yearly 
Savings and the model utilizes constant yearly savings. 
Although the formulas will result in different IRRs, any 
change in savings will result in an equi-proportional change 
im rate of return. 

Additionally, sensitivity analysis will be performed on 
the model. Initial analysis will restrict itself to the 
Single variable case and will be expanded later to consider 
multivariable deviations. The purpose of this analysis will 
be to determine the level of change of input variables to 


effect a change in ranking. 


Dee CONTENTS 

The following chapter introduces the reader to DPPO and 
the Productivity Investment Fund (PIF). The background of 
DPPO and its functions, an overview of PECI's, the PIF's 
past funding levels, and procedures used to obtain these 
funds will be addressed. Chapter III will familiarize the 
reader with risk and uncertainty as well as techniques in 


dealing with each. It will further apply the use of 
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uncertainty to the PIF. Chapter IV defines the terms and 

gives a detailed description of the model. It will explain 

how branch values were obtained and discuss the basic 
assumptions of the model. In Chapter V, the ranking derived 
using uncertainty will be compared to the current ranking, 
sensitivity analysis performed, and the results of the analysis 
revealed. Chapter VI Gilles 2s this thesis and present 


conclusions and recommendations derived therefrom. 
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II. DEFENSE PRODUCTIVITY PROGRAM OFFICE (DPFPO) 


A. BACKGROUND 

As previously stated, emphasis on productivity and 
efficiency within the Department of Defense (DoD) began 
in 1969. Initial attempts called for the establishment of 
the Defense Economic Analysis Council in October 1972. This 
council served in an advisory capacity to the Assistant 
Secretary of Defense (Comptroller) and encouraged the appli- 
cation of economic analysis and program evaluation in order 
to increase the cost effectiveness of budget proposal inputs 
to the Planning, Programming and Budgeting System (PPBS). 
In August 1975, DoD Directive 5010.31 established the De- 
fense Productivity Program. The primary objective of the 
program is "to achieve optimum productivity growth (increase 
the amount of goods produced or services rendered in relation 
to the amount of resources expended) throughout DoD" [6: 
pg. 1}. This directive required Defense organizations to 
be both effective and efficient in the utilization of all 
types of fund resources (operating and investment) as well 
as labor resources. Furthermore, the directive identified 
productivity measurement, productivity enhancement, and 
productivity evaluation as key elements to the program. 
Although all three elements play important roles in the 
program's success, only Productivity Enhancement (PE) will 


be discussed in this paper. 


Bese 








In 1979 DoD Directive 5010.31 was reissued, forming the 
Defense Productivity Program Office (DPPO) which had over- 
all responsibility of the Defense Productivity Program (DDP). 
DPPO was placed under the cognizance of the Assistant Secre- 
tary of Defense (Manpower, Reserve Affairs and Logistics) 
(ASD, MR&L) in 1981. 

The DPP to be developed would include a planned approach 
to PE including, but not limited to, the development of 
Productivity Enhancing Capital Investment (PECI) programs. 
PECIs are classified as: 

1. Office of the Secretary of Defense (OSD) sponsored 
projects; 
2. Industrial Fund Fast Payback (IFFP) projects; and 
3. Other Component Sponsored Investment (CSI) projects. 
OSD sponsored projects include the following categories: 
a. Productivity Enhancing Incentive Fund (PEIF) projects. 


These projects are 


..efast payback PECI projects financed from drawing 
accounts established within annual appropriations. 
These projects cannot exceed $100,000 or cost 
limitations established by OSD (whichever is 
greater) and must amortize within 2 years of 

the date they become operational. [7: pg. 4] 


b. Productivity Investment Fund (PIF) projects. 
These are projects that have been 


...competitively selected by OSD from candidate 
proposals submitted by DoD Components and financed 
through traditional budget appropriation processes 
from funds set aside by OSD for this purpose. 

PIF projects must be expected to amortize within 

4 years of the date they become operational. 


[7: pg. 4] 
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Since this thesis will deal only with PIF projects, discussion 
on PEIF, IFFP and CSI projects will be omitted. However, 
PIF funding levels and procedures will be of interest later 


and will be the next topic. 


B. PRODUCTIVITY INVESTMENT FUND (PIF) FUNDING LEVELS 
To better understand the magnitude of these investments, 

a look at past and proposed funding levels may prove bene- 
ficial. The first PIFs were funded in 1981. Money for the 
program was $105 million that year and $110 million for 1982. 
David Whipple and Jack LaPatra [15] completed an evaiuation 
of DPPO's activities revealing, 

The average cost of PIFs has been $2 million, with 

an average payback of 2.5 years. Having an average 

lifetime investment of $11.2 million, they are 

expected to return $6 for $l invested. Approximately 

$700 million has been requested under PIF by all 

services for FY 83-87. [15: pg. V-15] 
The average cost of proposed projects for FY84 budgeting is 
approximately $1.6 million, slightly less than previous 


years. How a Service Component proceeds to obtain funding 


Zomewaeproductivity project is the next topic. 


C. PIF FUNDING PROCEDURE 

Department of Defense Instruction (DoDI) 5010.36 estab- 
lishes policy and prescribes procedures for the Productivity 
Enhancing Capital Investment (PECI) program with which the 
PIF is affiliated. Excerpts from this directive concerning 
Pemrcy indicate the intent of the program by stating, "PECI 


funding procedures focus upon financing those projects that 
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ellestcltuee Capital for labor" {W: pg. 2]. “in yet another 
section, guidelines for project selection is delineated, 

Top priority will be given to those potential invest- 

ments that amortize in the shortest period of time 

and those with the highest potential internal 

rate of return on investment or the highest net 

present value. [7: pg. 2] 
Furthermore, the instruction dictates a specific format for 
DoD Components to follow when requesting money for produc- 
tivity projects. A copy of the format is contained in 
Reference 7. Some of the required information that will 
be of particular interest later includes total cost, total 
savings, internal rate of return (IRR), savings to investment 
ratio, rate of investment per manpower space and cost-benefit 
streams. These concepts are explicitly defined and evaluated 
in Chapter IV below. From this base, net present value (NPV) 
and payback period (P-P) can be calculated and economic 
analysis performed. For purposes of uniformity, the term 
savings to investment ratio is synonymous with return on 
investment (ROI). 

The sequence of events from project initiation to project 

funding may be outlined as follows: 

1. A DoD Component prepares an initial proposal for a 
project categorized as a PIF using the format af [QDI 
poo. 36. 

2. This proposal is forwarded via the appropriate chain 
of command for approval. 


3. Each service then compiles a iist of the "best" proj- 


ects, normally in rank order, that meets its needs and objectives. 
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4. These lists are then forwarded to DPPO who conducts 
an economic screen. The screening process consists of 
recomputing the information and checking the accuracy of 
each proposai in terms of calculations and validity of 
projections. | 

5. The projects are separated according to function, 
i.e., Automatic Data Processing (ADP), aircraft maintenance, 
etc., and forwarded to the appropriate Office of the Secre- 
tary of Defense (OSD) Functional Manager for a final screening. 

6. Projects are returned to DPPO who computes a final 
ranking of all approved projects. This ranking is based on 
three key indicators: IRR, ROI and Investment Per Manpower 
Space Saved (INVPERSV). Each indicator receives equal weight 
with ties in rank going to the project with the highest IRR. 

[mernrom the current budget set aside for PIF projects, 
all out-year funding on prior approved projects is deducted, 
1.e€., a FY8l1 project requiring funds for two years receives 
its FY82 money from the FY82 budget. A running cumulative 
sum of investment costs on currently approved projects is 
performed and the balance of PIF monies allotted. It should 
be noted that not all projects approved will be funded. 

8. DPPO then issues a Program Budget Decision (PBD) to 
the services who, in turn, add this money to their respective 
Service Budget requests. 

9. Once Congressional approval is received, as it usually 


is, the funds are appropriated accordingly. This entire 
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cycle takes approximately 1 year to complete and any project 
not funded that year may reapply the following year. 

At this point, the reader should have a basic understand- 
ing of DPPO, PIF and the procedures used to obtain project 
funding. What may not be so clear is the role that risk 
and uncertainty play. The next chapter attempts to intro- 


duce these terms and relate them to the PIF. 
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IIIT. RISK AND UNCERTAINTY 


A. INTRODUCTION 

The word decision may be defined as a choice among 
alternatives. In any investment, a decision may be made 
under conditions of certainty, risk, or uncertainty. Cer- 
tainty postulates that the decision maker knows in advance 
all parameter values that will affect the decision. With 
risk, he is aware of all future states that will affect his 
decision and can place a probability distribution on the 
value of the occurrence of each state, i.e., the probability 
distribution which describes possible outcomes is known. 
According to Morris [13], uncertainty implies that a decision 
maker may or may not be aware of all possible states and may 
Or may not be able to place a probability distribution on the 
occurrence of each. 

Figure 1 depicts a decision problem represented by a 
payoff matrix. Here, the columns, S)1Sor++-+/S., represent 
future states of nature and the rows Ap raneee +r, are the 
alternatives. The SY Wheto = ieee. ml and je= 1,2, <2 2,n, 
represent the payoff of alternative 1 in state j. Ifa 
probability distribution can be placed on the states of 
nature, then Morris [12], Fishburn [8], Luce [10] and Savage 
[14], give several principles that may be used to aid in 


decision making. These principles, expectation, most probable 


Ue, 








Figure 1. Typical Payoff Matrix 


future, expectation-variance, and aspiration level, will be 
discussed in detail in Chapter IV. If the decision is made 
under uncertainty, then the Laplace Principle, minimiax or 
maximin, minimin or maximax, Savage's minimax regret, and 
the Hurwicz principle are common principles of choice. 

Clark, et al [4] explain another technique in dealing 
with risk and uncertainty known as Utility Theory. This 
is an attempt to formalize rational decision making. In 
this approach, preferences among alternatives are specified 
by the decision maker. The utility value attached to the 
various alternatives then represent all aspects that are 
relevant to the decision. 

If a decision maker perceives different levels of risk 
associated with the future states, then the Certainty 
Equivalent Method may prove useful. This approach permits 
adjustment for risk by incorporating the decision maker's 
utility preference for risk. In an economic scenario, it 


is reasonable to assume that estimates of cash flows are 
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likely to be more accurate during the early periods of an 
investment than in the later years. Subsequently, the risk 
should be adjusted to reflect this. In dealing with simi- 
lar situations, whether economic or not, the certainty 
equivalent method may be preferred. 

Raiffa [13] explains a popular decision technique known 
as a decision flow diagram or "tree." Here, a manager has 
several courses of action he may take. For each course of 
action, there may be several consequences associated with 
it. Furthermore, with each consequence there exists a 
probability of that event occurring. Figure 2 depicts a 
simple example of a decision tree. In this example, a deci- 


sion maker has a choice of investing $15,000 on a project. 


t 
DON 't INVEST $0 


Unsuccessful (0.2) -$10, 000 

argqinall Lat. 5 
aa Cc rgina Successfu $20,000 
Ve Successful (0.3 $60 ,000 


Figure 2. A Simple Decision Tree 


If he decides not to invest, he gains or loses nothing. If 
he invests, then the project will be unsuccessful with 
probability 0.2, marginally successful with probability of 


0.5 or highly successful with probability 0.3. In this 
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particular case the decision maker faces an 80 percent chance 
of a successful project. Taking this one step further, the 
principle of expectation or expected value can be used. 


Mathematically, expected value, R is 
_ N 
R = )eReP: e5 


where: 


R is expected value, 
R. is the value of the ith outcome, 


P. is the probability the ith outcome occurs, 


N is the total number of outcomes. 


Applying this to the decision tree yields 


R (-$10,000 x0.2) + ($20,000 x0.5) + ($60,000 x 0.3) 


92.6700 0, 


Depending upon the decision maker's risk posture, l1.e., the 
Minimum return he will accept on this investment, he may or 
may not invest. 

It should be noted that the probabilities associated with 
the occurrence of consequences may not be very easy to obtain. 
When reviewing the available courses of action, the decision 
maker always has the option of delaying a decision until 


further data can be gathered which might provide insight 
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into the results of the various choices. However, delaying 
a decision usually has an associated cost. This cost may 
be in terms of time, money, workforce or numerous other 
things. According to Mooney [11], significant in this area 
are the benefits derived from increased information and the 
imecrease in certainty as a function of time. Figures 3 and 
4 graphically illustrate this problem. 

The dilemma of determining how much information is cost 
effective can often be viewed as a problem itself. How one 
obtains information can help in reducing costs. If data is 
readily available, the tools of regression, data analysis, 
non-parametric statistics and probability theory can be use- 
ful in providing a structured solution. On the other hand, 
if facts must be gathered, then other techniques may be 
used as well. Moody [11] suggests several non-mathematical 
means available including consensus thinking, brainstorming, 
the delphi principle, fish bowling, didactic interaction and 
collective bargaining. In addition to the above, other methods 
such as PERT (Program Evaluation and Review Technique) and 
QUID (Quantified Intrapersonal Decision Making) are becoming 


readily accessible. 


Bree PLECATION OF RISK AND UNCERTAINTY TO THE PIF 

During the funding process for Productivity Investment 
Funds (PIF), each project is checked for accuracy in its 
projections. .Components requesting funds review and analyze 


past data on costs, workloads, etc., in an attempt to forecast 
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Costs and benefits 


Uncertainty and cost of 


increase 


gathering data increase 


Cost of 
gathering data 





Time 


Figure 3 Cost-Benefit Time Curve 


Cost of gathering 
data 


Uncertainty 


Time 


Figure 4 Cost-Uncertainty Time Curves 
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costs and savings into the future. These savings are the 

key to calculating such indicators as internal rate of return 
(IRR), return on investment (ROI), and investment per man- 
power space saved (INVPERSV). However cautious and meticu- 
lous an analyst or manager may be, these savings may or may 
not be realized. Techniques of risk and uncertainty may 

aid in taking futuristic inputs into account and better 
predicting savings. An example might best portray this 
statement. 

Consider a machine that extracts rivets from aircraft 
wings at a rework facility. The investment cost of the 
machine is $60,000. Since the cost incurred is in the very 
near future, it will be assumed to be accurate (providing 
installation costs, transportation costs, etc., have been 
incorporated). In the cost-benefit stream, savings are 
forecast to be $30,000 per year for 10 years. Furthermore, 
assume the projected savings were based on a constant work- 
load, that of the current level, throughout the economic 
life of the machine, i.e., 10 years. Clearly such factors 
as war, depression or recession, the introduction of a low 
maintenance aircraft, etc., could substantially increase 
or decrease the workload of the facility and, hence, pro- 
jected savings. The list of factors that could affect the 
projections 1s practically endless. To construct a model 
incorporating all of these factors would be tedious and costly, 


with the model becoming unmanageable. If risk and uncertainty 
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were used on a macro level, most of these problems would be 
alleviated. (The word macro is used here to distinguish the 
analysis from a micro aspect.) The micro aspect of analysis 
would consider each factor, to achieve the final result. In 
the macro sense, one is concerned only with whether or not 
realized savings were above, below, or as projected without 
concern for the factors that would cause it to deviate. The 
macro model could enhance managability and still yield 
workable results, especially if not one, but 50 to 100 projects 
had to be dealt with. 

The impact of risk and uncertainty in dealing with PIF 
projects should be apparent. The methodology and model used 
toward this end is presented in following chapters. It 
should be noted that the procedure used is not exclusive to 
investments alone, nor is it the only method applicable for 
analysis. Variations on this theme will also be mentioned 


later. 
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IV. AN ECONOMIC MODEL 


A. DEFINITIONS 

The underlying motive of any business is to make a profit. 
One method of achieving this is through capital investments. 
It follows that managers expect money invested today to in- 
crease in amount as time passes because they expect to earn 
a profit on that investment. To the manager, therefore, the 
value of money today is more than its value at some future 
time. Anthony and Reese [l] make an interesting analogy, 

"An investment is thus the purchase of a future stream of 
expected cash inflows" [2: pg. 710]. Cash inflows are simply 
earnings or savings. 

If several investment alternatives exist, then a basic 
criterion must be established for the purposes of comparison 
and evaluation. Although five criteria are explained next, 
only three (IRR, ROI, INVPERSV) are used by DPPO. 

The first economic indicator, Net Present Value (NPV), 
is the difference in the present value of the benefits 
(Savings) and the present value of the costs at a given 
discount rate (the interest rate used to discount or calcu- 


late future costs and benefits so as to arrive at their 


present values). Mathematically, this may be expressed as: 
n Sy 
RENE Fa oS eremmceer = 3 er (2) 
t=1 (1+k) 


2/7 





where: 


Ay = present value of the cost of the project, 
Sy = savings received in period t, 

k = appropriate discount rate, 

2 = time period, and 

n = useful life of asset. 


Ee Ay 1s incurred over a period of time, then 


n A 
A — ) a al ’ ey) 


0 ea Cosa 


where A, is the cost during period t. If the NPV is posi- 
tive, it means the project is expected to yield a return in 
excess of the required rate. If it is zero, the yield is 
expected to equal the required rate. 

The discount rate that equates the present value of 
the future cash flows with the present value costs of an 
investment is known as the Internal Rate of Return (IRR). 


This is calculated by determining the discount rate that 


will make the NPV zero: 


n S 
F=-A, = 0, (4) 
tal) Ul + 5) 
where: 
r = IRR, 
Ss, = Savings received in period t, 
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t = time period, 
A = investment cost, and 
n = useful life of asset. 


If cash inflows are uneven, the trial and error method is 
recommended. Computers ease the tediousness of the compu- 
tation; however, DoD has derived a more simplistic procedure. 
Dividing the project cost by the average annual savings will 
yield a factor. This factor and the number of years in the 
economic life of the project can be used to enter a table 
and select the IRR. If the cash inflows are even, Anthony 
and Reese [1], using the same calculation, offer the IRR 
based upon the same table inputs. 

The Payback Period (P-P) refers to the number of periods 
required for the (undiscounted) cumulative cash inflows to 
have the same value as the investment cost. 

Another indicator, Return On Investment (ROI), is synony- 
mous with the savings to investment ratio. This method com- 
pares yearly income of a project with the investment in the 


asset [l: pg. 52]. The formula used by DPPO is: 


n 
ae t 
ROI = ray) 
} 
c 
t=1 ¢ 
where: 
S = savings received in period t 
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C, = costs incurred in period t 
ie = time period 
n = useful life of the asset. 


Finally, the Investment Per Manpower Space Saved 
(INVPERSV) is simply an indicator that compares cost to 
manpower. A manpower space is best defined by example. 

If two clerks are required to perform a task, and the 
introduction of a new system requires only one of then, 
then the other is freed and can be assigned elsewhere. This 


equates to saving one manpower space. 


B. THE MODEL 

Although the techniques of both risk and uncertainty 
have been discussed, the model that will be presented next 
will deal strictly with uncertainty. This is not tq say 
that the model could not be used with risk. After careful 
analyses of several dudit reports, there was not enough infor- 
mation to derive probability distributions, thus eliminating 
the introduction of risk. In particular, the audit reports, 
for the most part, failed to compare actual (or realized) 
costs or savings to that which was proposed. Without this 
information, it would be extremely difficult to obtain proba- 
bility distributions (the use of subjective probabilities 
could have been used). Further discussion on this point is 
contained in Chapter VI. 

In the process of evaluating and ranking PIF projects, 


DPPO used three indicators; IRR, ROI and INVPERSV. The 
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model, therefore, will contain three submodels used to 
evaluate each indicator respectively. A tree diagram was 
used in each submodel in order to present the different 
outcomes in an orderly fashion. Figures (5), (6) and (7) 
are graphical representations of these models. The under- 
lying assumption used in the development of each diagram is 
based on the independence of cost, savings, manpower space 
saved (MPS) and cash flow. The term independence is used 
to signify that an outcome of one variable is not dependent 
on any of the others. This assumption allows one to branch 
out the different future states of nature and then apply 
uncertainty techniques. 

It should be noted that the IRR model utilizes one branch 
instead of two as represented in the other models. This was 
done for two reasons. The first is that Sciortino's [1] 
audit of FASCAPsS (Fast-Payback Capital Investments) revealed 
that 11.7 percent of all funded projects resulted in cost 
overruns of 10 percent of the total projected cost. This 
sum did not appear to be significant in comparison to the 
total cost of all of the funded projects. Secondly, the 
program used to evaluate the model was written in APL (A Pro- 
gramming Language). The IRR computation was derived through 
an iterative loop which, in APL, can be costly due to the 
limitations of APL's capabilities (one run of 15 projects 
takes approximately 10 minutes, equating to a cost of nearly 


$80.00). Comparing the cost of computaticn to the benefits 
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derived from the additional branching, it was felt that one 
branch was adequate. 

As previously stated, the main model was broken down 
into three submodels, one for each indicator to be ranked. 
Once the program has computed the branch values in each 
submodel, it proceeds to apply the techniques that deal with 
uncertainty, in particular, the MAXIMIN, MAXIMAX and LaPlace 
peenciples. 

The MAXIMIN principle may be viewed as a pessimistic 
approach.: This principle of choice suggests that the maxi- 
mum of the minimum gains be chosen, or select the best of 
the worst outcomes. In this section of the program, the 
minimum value of each submodel is selected and stored in three 
vectors (one for each indicator). These vectors are then 
placed ina Agi by 3 matrix where the rows represent projects 
and the columns are the respective indicators. A value in 
the matrix is the minimum value for an indicator peculiar to 
a project. Once this matrix is complete, each column (or 
indicator) is ranked from highest to lowest (i.e., the great- 
est number receives rank one, and so on). In the event of 
a tie, an average rank is used. These rankings are then 
placed in another matrix which will be referred to as the 
rank matrix. The ranks are then summed across the columns 
(indicators) and then re-ranked from lowest to highest. 


By summing across the columns, one is merely adding the ranks 


1 there are 47 PIF projects funded in FY84. These 
projects form the data base. 
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of the indicators for each project. Subsequently, the 
project with the lowest combined rank is the best project 
and hence the reason to rank from the lowest (rank one) to 
the highest. The result of this procedure is a vector of 
47 elements (one for each project) representing the final 
ranking. 

The second technique used is the MAXIMAX principle. 

This is an optimistic approach where the best possible out- 

come is maximized. The procedure used here is identical 

to the MAXIMIN computation with the exception that the maxi- 
mum value in each submodel is chosen instead of the minimum. 

The LaPlace principle is computed somewhat differently. 
This method assumes that the branch values are equally likely 
to occur within each submodel. Using this probability dis- 
tribution, an expected value for each indicator can be de- 
rived. The result is a matrix of expected values for each 
project and indicator. This matrix is then ranked using 
the same procedure as previously discussed. 

The ultimate product of the program is a matrix of the 
final rankings of each product using each of the three 
methods of uncertainty. It is now possible to judge the 
effect of uncertainty by comparing these rankings to those 
derived using DPPO's procedure. However, before these 
results are revealed, some discussion on software support 
for the model and problems that arose during the develop- 


ment of the model warrant attention. 
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C. SOFTWARE SUPPORT 

In the process of evaluating the model, two data files 
were created and several functions and programs written. 
The first file, THESISD, is a 47 x5 matrix where the rows 
represent the projects and the columns are total cost, total 
savings, manpower spaces saved, NPV, and economic life. 
CFMATRIX is the second file containing the cash flows for 
each project over a period of 28 years. To ensure matrix 
conformability, zeroes had to be added to those projects 
whose economic lives were less than 28 years. These zeroes 
are removed in the main program to reduce computations. 

The main program, UNCERT, is written in APL and calls 
its data from the two files. It internally computes ROI 
and INVPERSV branch values, selects max and min values and 
calls other functions for IRR values (CALIRR) and to perform 
rankings (RANKUP, RANKDN). A listing of UNCERT as well as 
the functions are contained in Appendix (A). Copies of the 


base data composes Appendix (B). 


D. PROBLEM AREAS 

In the development and documentation of the model, 
several problems were encountered which substantially altered 
the procedure to be used. 

The first difficulty concerned itself with the INVPERSV 


computation. The formula used by DPPO can be expressed as 


_ Total Gost 
ee Manpower Spaces Saved (MPS) ° ae 
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This presents two problems. First, if MPS is zero, INVPERSV 
is undefined (an MPS of zero is not uncommon). To correct 
this deficiency, DPPO arbitrarily sets INVPERSV equal to 
9999.99. Secondly, if MPS is between zero and 1, then 
INVPERSV is greater than the total cost. Currently, there 
is no correction for this. Another difficulty with this 
indicator lies in its lack of logical attractiveness. Ex- 
plicitly, IRR and ROI are benefits which implies bigger is 
better. INVPERSV, as presently used, is a cost (bigger is 
bad). Although the procedure used by DPPO takes this into 
account, it is felt that the indicator contradicts the logi- 
cal flow when used with the other indicators. 

A second problem arose when trying to duplicate DPPO's 
calculations of IRR. Using Equation (4) and assuming the 
cash inflows were present value (verified by DPPO to be 
correct), the IRR's obtained from CALIRR differed from those 
calculated by DPPO. 

Another minor discrepancy involved the value for economic 
life. THESISD contained one value, however, when calculating 
IRR, different economic lives were obtained using the cash 
inflow information. This had no effect on calculations 
Since IRR used strictly cash flow information. 

Concern was additionally raised due to the fact that 
DPPO's final ranking could not be reproduced. Further dis- 
cussions on this point are unnecessary since modifications 
to the analysis to be performed would create new rankings, 


thus eliminating any reason to duplicate the original ranked list. 


38 





E. PROCEDURE 

At this point the reader should have an understanding 
of the model, how uncertainty will be introduced, and some 
problems that were encountered in the process. This section 
will: a) discuss how the INVPERSV problem was solved, 
b) explain how deviation levels were obtained, c) how the 
IRR confusion was treated and d) given an outline for 
sensitivity analysis that will be performed. 

The first topic is how to correct the INVPERSV problem. 
If the original indicator yielded a cost, then the inverse 
of the indicator would be a benefit. Additionally, the in- 
verse would also alleviate the problem of dividing by zero 
or a fraction. However, the inversion of the indicator re- 
sults in a decimal. By scaling the result (multiply by 1000), 
the indicator becomes more readable and comparable to the 


other indicators. This transformation 1s given by: 


INVPERSV = (Fotal Manpower Spaces Saved, x 1000 


Total Cost . (7) 


Take, for example, an INVPERSV of 275.0. The inverse would 
be 0.0036. Scaling by 1000 yields 3.6. Several things 
should be noted. First, using a scalar of 1000 is purely 
arbitrary. Any scalar could have been used. Secondly, by 
inverting the indicator, the ranking must be done from high- 
est to lowest. This did not change the original ranking 


done by DPPO. The final point is that by inverting, the 
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logical attractiveness of INVPERSV is achieved. The reason 
for scaling is discussed more in Chapter VI but deals with 
an alternate means of obtaining a ranking. 

In Section C, the idea of changing different variables 
fewe,, COsSt, Savings, etc.) was introduced. How much to 
vary these variables is what is of interest here. In the 
program, deviation levels are represented as a vector with 
three elements. These elements represent variables above 
the projected level, as projected, and below projected 
levels. That is to say, if a cost overrun were to occur, 
the average overrun would be, for example, 20 percent of 
the projected cost. This can be mathematically expressed 
as, cost overrun = 1.2 x projected cost. Similarly, to main- 
tain the projected cost, a multiplier of l is used while a 
number less than 1 would relate a cost underrun. Deviation 
levels can be determined by 

Cost Deviation = Borner ed GSE : (8) 

By using Sciortino's [1] audit and Equation (8), deviation 
levels for cost, savings, and cash flows were obtained. Man- 
power variations, however, were purely subjective. [In 
reviewing several audit reports, there is no data to address 
this point. The deviation levels used to obtain variations 
were 1.1 for a manpower overrun and 0.9 for an underrun. The 
sensitivity analysis that will be performed later will further 


address this area. 
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IRR fluctuations created an area of concern since they 
could not be duplicated and since the results obtained using 
Equation (4) showed no correlation to DPPO's data. To 
remedy the situation, all 47 projects were ranked using 
Equation (4) to calculate IRR and Equation (7) for INVPERSV 
(ROI calculations were the same). The results of this run 
will be referred to as the DPPO Base since this is what the 
rankings should have been with all problems aside. Next, 

a run was made to introduce uncertainty. These results were 
compared to the DPPO Base to see if any significant changes 
had taken place. For reference, Appendix (C) is the DPPO 
Base case and Appendix (D) displays the results after uncer- 
tainty was applied. 

In order to examine which variables and uncertainty 
methods were critical, sensitivity analysis was performed. 
The variables of concern are cost, savings and manpower spaces 
saved. Initially, single variables were used, followed by 
changes in two variables and finally, by three. Appendix (E) 
contains the results of these runs while the next chapter 


discusses this issue in greater detail. 
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Ve ReoUETS 


A. INITIAL COMPARISON 

DPPO's ranking was computed using the new INVPERSV and 
IRR ranking. The base case, which introduces uncertainty 
into the ranking, was run using deviation levels outlined 
in Table I. These values were obtained using Sciortino's 
[1] audit and Equation (8). The output of the two runs were 
then compared to determine if the base case had any impact 


on DPPO's ranking. 


TABLE I 


Deviation Levels Used in the Base Case 


Above Below 
EEOjecred Peg ceed 
Cost 1.256 0.825 
Savings/ 
Cash Flows 1.014 Ors 6 
Manpower 
Spaces decks 0.9 


Before any comparison may be undertaken, it might be 
beneficial to define how this impact is to be measured. Three 
terms are used to judge the magnitude of difference between 
DPPO's ranking and the base case; 1) significant (S) implies 


that 5 or more projects experienced any rank jumps or any 
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two projects had jumps of 5 ranks or more, 2) slightly 
feet icant (SS) is defined as 3 or more projects had rank 
jumps or any two projects had jumps of three ranks or more, 
and 3) no significance (NS) merely represents any situation 
less than those described above. 

Initially, all 47 projects were compared. This revealed 
that the MAXIMIN and LaPlace Principles significantly changed 
the rankings while the MAXIMAX showed little to no change. 
DPPO's rankings were divided into three groups; projects 1 
through 15, 16 through 31 and 32 through 47. The reason for 
subdividing was to observe where the jumps in ranking 


occurred. A summary of the results is contained in Table II. 


TABLE II 


Summary of Initial Comparison 


Project 

Number MAXIMIN LaPlace MAXIMAX 
All S S NS 
1-15 Ss NS NS 
16-31 s SS NS 
32-47 S S NS 


Beamg this information, it is evident that jumps occur more 
frequently in the lower projects. The reason for this 
occurrence is primarily due to the first 15 projects offering 


Significantly greater IRRs, ROIs, and INVPERSVs than the 
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others. This information will play a role in performing 
sensitivity analysis and therefore the discussion will be 


delayed until then. 


B. SENSITIVITY ANALYSIS 

Due to cost constraints in running all 47 projects, 
only the first 15 were used to perform sensitivity analysis. 
To begin, a new base case dealing with uncertainty was derived 
and would be used to compare 24 variant runs. The purpose 
of this analysis was to determine which variables were 
critical and which uncertainty methods were sensitive to 
the changes. First, single variable deviations were analyzed, 
followed by two and three variable combinations. 

The results of the initial runs with single variables 
indicated that all three methods were insensitive to minor 
changes in the variables with one exception. This run dealt 
with increasing the bounds (above projected and below pro- 
jected deviation levels) of savings by 20 percent and creating 
Significant rank jumps in the MAXIMIN ranking and slightly 
Significant jumps in that of LaPlace. 

The deviation levels were then preset to values outlined 
am Table III. It should be noted that these levels are 
substantial deviations from the projected values. This is 
primarily due to the nature of the first 15 projects. As 
mentioned in Section A of this chapter, the initial compari- 
son noted little change in ranking among the three methods. 


This is attributable to the large variations in each of the 
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TABLE III 


Deviation Levels Used in Sensitivity Analysis 


Above Piso" Ska Below Peoieceed 


Cost haya G72 Or 2 
Savings/ 

Cash Flows 2.0 Dau 
Manpower Spaces Ore OF 2 


variables in the first 15 projects as compared to the rest. 
These variations are presented in Table IV. Due to restric- 
tions in programming, other projects could not have been 
selectively chosen without redrafting files. However, despite 
this problem, it is still possible to determine critical 
variables and sensitive methods. 

Table V conveniently summarizes the results obtained from 
the analysis. A word on notation though, might aid in its 
understanding. In the description column, the variable being 
changed is described as being either high or low. High refers 
to an increase in the above projected deviation level only 
while low, decreases the below projected level. Run 2b 
decreased both bounds by a set percent and Run 3b increased 
both bounds. As stated earlier, other runs utilizing this 
procedure yielded no changes with the exception of 3b and, 
therefore, eliminated further use of the procedure. 

To further summarize the contents of Table V, it can 


be said that the LaPlace principle was the most sensitive to 
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change. This may be attributed to the cost-savings rela- 
tionship and, in particular, savings deviations. This would 
affect both the ROI and IRR. The method was insensitive to 
manpower variations. The only other method to show a reac- 
tion was MAXIMIN. This is strongly evident in low savings 
cases. Low savings (implying low cash flows) will, in turn, 
drive IRR with an additional effect on ROI. Once again, 
manpower fluctuations had little to no effect. The last 
principle, MAXIMAX showed little to no change during the 
course of the analysis. 

To summarize, the LaPlace principle showed sensitivity 
to more variables or combinations thereof. The MAXIMIN 
method was the only technique to cause significant jumps 
in rankings. With both methods, savings seemed to be the 
critical variable indicating that IRR is the main driver in 
the rankings, followed by ROI. INVPERSV had little to no 


effect in the rankings. 
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VI. CONCLUSIONS AND REMARKS 


A. CONCLUSIONS 

Since the objective of this thesis was to determine if 
the methods of risk and uncertainty could change project 
rankings from the present procedure, this point shall be 
addressed first. It is apparent that both the MAXIMIN and 
LaPlace principles will change the ranking and their incor- 
poration into the present procedure should be investigated 
as a short term solution. Additionally, the LaPlace approach 
is recommended above the others for several reasons: It is 
not an extreme selection; it makes more use of the different 
outcomes than the other methods; and it is more sensitive 
to the variables and combinations thereof. 

This paper does not purport to advocate strict adherence 
to methods of uncertainty in decision making. The use of 
risk is a better solution to solving problems of this nature. 
To enable the use of risk techniques, it is recommended 
that audit reports compare realized costs, savings, and 
manpower to that which was projected. Furthermore, since 
DPPO maintains a data base from which the projects are ranked, 
this data base should also serve as a reference in doing 
audits. As mentioned before, audit reports should also 
address manpower spaces, which to this point, has gone 


without remark. 
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The use of the new INVPERSV and IRR equation have been 
discussed sufficiently. Their incorporation into the 
present procedure is highly recommended. 

The use of equal weights for the indicators is another 
questionable area, and will be addressed in the final section 
of this chapter. 

Although more conclusions could be drawn, the next 
section will introduce a corrective procedure which would 


provide better and more accurate rankings. 


B. REMARKS 

The content of this thesis introduced the use of risk 
and uncertainty into DPPO's procedure. This is what it was 
commissioned to do; however, a better approach utilizing 
multiattribute utility theory may prove to provide quite 
different results. 

The present procedure has several flaws. The first is 
using linear combinations of rankings to obtain a final 
aggregate ranking. Economic indicators are measures of 
effectiveness (MOEs). These MOEs have a common origin 
(zero) and are measured on a ratio scale, meaning a state- 
ment may be made to the effect that one IRR is twice as 
good as another IRR. 

The use of ranks, changes the scale to ordinal and this 
information is lost. An example may best illustrate this 
point. Suppose a comparison of projects A and B is to be 


made. Project A has an IRR of 200 percent, an ROI of 87 


a0 





percent and an INVPERSV of 78. Project B, on the other 
hand, has an IRR of 25 percent, an ROI of 88 percent and 
an INVPERSV of 79. According to the present procedure, 
project B is better. However, if a ratio scale is used, 
then project A would be the wiser choice since A's JESSE 
is 8 times greater than B's with the other indicators about 
equal. 

Additionally, the sensitivity analysis pointed out that 
IRR seemed to drive the rankings with ROI having a smaller 
input. This indicates that the use of equal weights may 
not be as reasonable as initially thought. 

This topic should be considered an area for future study. 
The benefits derived from it could be substantial and lead 
to a more effective manner to judge projects and eventually 


realize their benefits. 
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APPENDIX A 


SOFTWARE SUPPORT 


This appendix contains the programs used to introduce 


uncertainty to the Productivity Investment Fund. 
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APPENDIX B 


DPPO'S DATA FILE 


This appendix contains the data used to perform the 


analysis. 
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Cash Inflows (Cont) 
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Cash Inflows (Con'*t) 
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Cash Inflows (Con't) 
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Cash Inflows (Con't) 
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APPENDIX C 


DPPO'S DATA BASE 


This appendix contains the results from the DPPO 


Base case. 
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DPPO Base Case (Con*t) 


PROJECT NUMBER:12 


760 3760 3760 3760 3760 3760 3760 3760 3760 3760 


LAPLAGE fi pide aon ROI IRR AND MPS 


PROJECT NUMBER:13 


9 459.9 459.9 459.9 45 


COOr sw 


69 


CR ROI IRR AND MPS 


RCO 


FROJECT NUMBER: 14 


4 191.4 191.4 191.4 19 
191.4 


mM 


- 


NOAM eo 
ee ee 08 eg t™ 
ee ata} 

mn YH = 
|. 3 ie ab “ 
Fe ee 
MAHON 
<u> Eo 
HO aceO 
mOnNR As 
aq oe eo 
Peto = 
—G xe Ay 5 
Rieti 
WOO ci ct 
eee ene 


—_ 


IRR AND MPS 


ES FCR_ ROI 


mI 





Geore S Sta SG Slew S Ste S SIC S)Slc¢ 


6°6hl 6°6hl 6°6hL 6°6hL 


6°6nt 6°6nl 6° 6hL 


SdW ONY WHT TOU od SANTY 


lL 
SdW GNW Yur IO 
6°6nl 6°6hlt 6°6 

6 
L 


saw anv wal 48a 
L 


e 9PCoU tC SEED f 9RtG te 9lt6 €,OPGGee 9lCome. 


0 aC. 


LSOD T 


Or 


Sita a5 vitae 


SMOTT HSWO 
GaAVS YAMOdNYW 
SONIAWS ‘IY LOL 
LSOO TWLOL 
>-SHTAVIATA LOAANT 


GL=HHEWNN LOALOUd 


(3,u05) asen eseg Oddd 


70 





DPPO Base Case (Con* t) 
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DPPO Base Case (Con* t) 
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This appendix contains the results of the initial 
runs used to determine the impact of uncertainty on DPPO's 


current procedures. 
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APPENDIX E 


SENSITIVITY ANALYSIS RESULTS 


This appendix contains the different computer runs 
performed during sensitivity analysis. Descriptions of 


the runs are noted at the top of each output. 
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